data <- read_csv("/Users/jorgevargasmutizabal/Desktop/Frog game statistics/R Analysis Frog/IN/nat.csv")
## Rows: 22404 Columns: 8
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## dbl (7): Player_ID, LexTale, Game_Version, Game_Level, Phrase_Condition, Que...
## lgl (1): Answer
## 
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
str(data)
## spc_tbl_ [22,404 × 8] (S3: spec_tbl_df/tbl_df/tbl/data.frame)
##  $ Player_ID       : num [1:22404] 1 1 1 1 1 1 1 1 1 1 ...
##  $ LexTale         : num [1:22404] 8 8 8 8 8 8 8 8 8 8 ...
##  $ Game_Version    : num [1:22404] 1 1 1 1 1 1 1 1 1 1 ...
##  $ Game_Level      : num [1:22404] 1 1 1 1 1 1 1 1 1 1 ...
##  $ Phrase_Condition: num [1:22404] 1 1 1 1 1 1 1 1 1 1 ...
##  $ Question_Num    : num [1:22404] 0 1 2 3 4 5 6 7 8 9 ...
##  $ Answer          : logi [1:22404] TRUE TRUE TRUE TRUE TRUE TRUE ...
##  $ Reaction_Time   : num [1:22404] 1.99 0.8 1.8 0.77 0.82 0.86 0.82 0.85 0.78 1.06 ...
##  - attr(*, "spec")=
##   .. cols(
##   ..   Player_ID = col_double(),
##   ..   LexTale = col_double(),
##   ..   Game_Version = col_double(),
##   ..   Game_Level = col_double(),
##   ..   Phrase_Condition = col_double(),
##   ..   Question_Num = col_double(),
##   ..   Answer = col_logical(),
##   ..   Reaction_Time = col_double()
##   .. )
##  - attr(*, "problems")=<externalptr>
data <- data %>%
  mutate(
    Player_ID = as.factor(Player_ID),          # Convert to factor
    LexTale = as.numeric(LexTale),              # Convert to numeric
    Game_Version = as.factor(Game_Version),     # Convert to factor
    Game_Level = as.factor(Game_Level),        # Convert to factor
    Phrase_Condition = as.factor(Phrase_Condition), # Convert to factor
    Question_Num = as.numeric(Question_Num),    # Convert to numeric
    Answer = as.numeric(Answer),                 # Convert to factor
    Reaction_Time = as.numeric(Reaction_Time)   # Convert to numeric
  )

# Verify changes
str(data)
## tibble [22,404 × 8] (S3: tbl_df/tbl/data.frame)
##  $ Player_ID       : Factor w/ 38 levels "1","2","3","4",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ LexTale         : num [1:22404] 8 8 8 8 8 8 8 8 8 8 ...
##  $ Game_Version    : Factor w/ 4 levels "1","2","3","4": 1 1 1 1 1 1 1 1 1 1 ...
##  $ Game_Level      : Factor w/ 8 levels "1","2","3","4",..: 1 1 1 1 1 1 1 1 1 1 ...
##  $ Phrase_Condition: Factor w/ 4 levels "1","2","3","4": 1 1 1 1 1 1 1 1 1 1 ...
##  $ Question_Num    : num [1:22404] 0 1 2 3 4 5 6 7 8 9 ...
##  $ Answer          : num [1:22404] 1 1 1 1 1 1 1 1 0 1 ...
##  $ Reaction_Time   : num [1:22404] 1.99 0.8 1.8 0.77 0.82 0.86 0.82 0.85 0.78 1.06 ...

Model 1: Simple Random Intercept

Formula: Reaction_Time ~ 1 + Question_Num + Phrase_Condition * Game_Version + (1 | Player_ID)

Can Answer: - How do reaction times generally relate to Question_Num, Phrase_Condition, and Game_Version? - Are there overall differences in reaction times due to the interaction between Phrase_Condition and Game_Version? - What is the baseline variation in reaction times across different players?

Cannot Answer: - Does the effect of progressing through questions vary among individual players? - Do individual players react differently to various phrase conditions? - How do individual differences affect the interaction between phrase conditions and game versions?

Model 2: Adding Random Slope for Question Number

Formula: Reaction_Time ~ 1 + Question_Num + Phrase_Condition * Game_Version + (1 + Question_Num | Player_ID)

Can Answer: - All questions Model 1 can answer. - How does the impact of Question_Num on reaction times vary across individual players?

Cannot Answer: - Do different players have distinct reactions to the various phrase conditions, beyond just a baseline reaction time difference? - How do individual differences interact with the combined effects of phrase conditions and game versions?

Model 3: Adding Random Slope for Phrase Condition

Formula: Reaction_Time ~ 1 + Question_Num + Phrase_Condition * Game_Version + (1 + Phrase_Condition | Player_ID)

Can Answer: - All questions Model 1 can answer. - How does the impact of different Phrase_Conditions on reaction times vary among individual players?

Cannot Answer: - How do individual differences in reaction to questions over time affect reaction times? - Does the relationship between phrase condition and game version differ by player, considering their specific reaction to both conditions and questions over time?

Model 4: Combining Random Slopes for Both Question Number and Phrase Condition

Formula: Reaction_Time ~ 1 + Question_Num + Phrase_Condition * Game_Version + (1 + Question_Num + Phrase_Condition | Player_ID)

Can Answer: - All questions that Models 1, 2, and 3 can answer. - How do individual differences in reaction to both Question_Num and Phrase_Condition interact to affect reaction times? - Does the way a player’s reaction time changes with questions relate to how they react to different phrase conditions?

Cannot Answer: - Specific inquiries about third-level interactions involving game version beyond the fixed effects structure, unless further random slopes for Game_Version or its interaction terms are included. - Questions about more complex nested or crossed random effects not specified in the model (e.g., random slopes for Game_Version, or cross-level interactions such as Phrase_Condition:Game_Version | Player_ID).

Summary

Each model’s ability to answer specific questions is tied to the random effects it includes. As you add more complexity to the model (like in Model 4), you can address more detailed and specific questions about individual variations and interactions between the effects. However, each addition also requires more data and computational power, and the risk of overfitting or convergence issues increases. Therefore, selecting a model should balance the need for detail with the practicality of fitting the model and the quality of the data available.

Let’s fit some models!

0. What is the average reaction time marginalizing over everything?

Oa: rt ~ 1 + (1 | participant)

Fitting this model allows you to determine how much variation in reaction times is due to individual differences between players versus random noise or other factors. A significant Player_ID variance component suggests individual differences are important; if it’s close to zero, reaction times are consistent across players, and the random effect may be unnecessary.

model1 <- lmer(Reaction_Time ~ 1 + (1 | Player_ID), data = data)

# Obtain and print a summary of the model to examine fixed and random effects
print(summary(model1))
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: Reaction_Time ~ 1 + (1 | Player_ID)
##    Data: data
## 
## REML criterion at convergence: 64205.2
## 
## Scaled residuals: 
##    Min     1Q Median     3Q    Max 
## -1.470 -0.389 -0.111  0.200 46.838 
## 
## Random effects:
##  Groups    Name        Variance Std.Dev.
##  Player_ID (Intercept) 0.04523  0.2127  
##  Residual              1.02245  1.0112  
## Number of obs: 22404, groups:  Player_ID, 38
## 
## Fixed effects:
##             Estimate Std. Error      df t value Pr(>|t|)    
## (Intercept)   0.9557     0.0352 37.1878   27.15   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#Diagnostic plots
plot_residuals_diagnostics(model1)

## `geom_smooth()` using formula = 'y ~ x'

# Model diagnostics
check_model(model1)

Random Effects Player_ID: The variance for random intercepts attributed to Player_ID is 0.01319 with a standard deviation of 0.1149, indicating small variability in reaction times among players compared to residual variability.

Residual Variance: The residual variance is 0.16394 with a standard deviation of 0.4049, showing that most variability in reaction times is due to unaccounted factors or inherent variability in individual reactions.

Fixed Effects Intercept (Overall Average Reaction Time): The intercept estimate is 0.83745 with a standard error of 0.01885, indicating an average reaction time of approximately 0.837 seconds. The t-value of 44.43 and a p-value of less than 0.001 confirm that this average reaction time is significantly different from zero.

Interpretation of the Model: The analysis shows small variability in reaction times due to individual differences (Player_ID) with most variance captured by the residual term. This suggests either consistent game difficulty across players or minimal variation in individual reaction times.

1. Does learning take place?

1b: rt ~ 1 + time + (1 + time | participant)

model2 <- lmer(Reaction_Time ~ 1 + Question_Num + (1 + Question_Num | Player_ID), data = data)
## Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
## Model failed to converge with max|grad| = 78.1395 (tol = 0.002, component 1)
## Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, : Model is nearly unidentifiable: very large eigenvalue
##  - Rescale variables?;Model is nearly unidentifiable: large eigenvalue ratio
##  - Rescale variables?
# Obtain and print a summary of the model to examine fixed and random effects
print(summary(model2))
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: Reaction_Time ~ 1 + Question_Num + (1 + Question_Num | Player_ID)
##    Data: data
## 
## REML criterion at convergence: 63625.1
## 
## Scaled residuals: 
##    Min     1Q Median     3Q    Max 
## -2.506 -0.381 -0.101  0.209 47.588 
## 
## Random effects:
##  Groups    Name         Variance  Std.Dev. Corr 
##  Player_ID (Intercept)  3.328e-01 0.576871      
##            Question_Num 3.768e-06 0.001941 -0.95
##  Residual               9.907e-01 0.995346      
## Number of obs: 22404, groups:  Player_ID, 38
## 
## Fixed effects:
##                Estimate Std. Error         df t value Pr(>|t|)    
## (Intercept)   1.0787300  0.0945331  7.1674459  11.411 7.44e-06 ***
## Question_Num -0.0007861  0.0003216  8.4820061  -2.445   0.0386 *  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation of Fixed Effects:
##             (Intr)
## Question_Nm -0.947
## optimizer (nloptwrap) convergence code: 0 (OK)
## Model failed to converge with max|grad| = 78.1395 (tol = 0.002, component 1)
## Model is nearly unidentifiable: very large eigenvalue
##  - Rescale variables?
## Model is nearly unidentifiable: large eigenvalue ratio
##  - Rescale variables?
#Diagnostic plots
plot_residuals_diagnostics(model2)

## `geom_smooth()` using formula = 'y ~ x'

# Diagnostic checks - Simulating residuals to validate model assumptions
residuals_simulation <- simulateResiduals(fittedModel = model2, n = 500)
plot(residuals_simulation)

# Model diagnostics
check_model(model2)

# Check for influential cases potentially affecting the model
#influence_measures <- influence(model3)
#plot(influence_measures, which = "cook")

# If assumptions are not met, consider re-fitting the model with transformations or different specifications
# model_revised <- update(model, . ~ . + log(Reaction_Time))
# print(summary(model_revised))

# Model interpretation - examining fixed and random effects
#fixed_effects <- fixef(model3)
#random_effects <- ranef(model3)

# Printing fixed effects for interpretation
#print(fixed_effects)

# Printing random effects for interpretation
#print(random_effects)

# Get estimated marginal means for specific values of Question_Num
emm_results <- emmeans(model2, specs = ~ Question_Num, at = list(Question_Num = seq(min(data$Question_Num), max(data$Question_Num), by = 10)))
## Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
## To enable adjustments, add the argument 'pbkrtest.limit = 22404' (or larger)
## [or, globally, 'set emm_options(pbkrtest.limit = 22404)' or larger];
## but be warned that this may result in large computation time and memory use.
## Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
## To enable adjustments, add the argument 'lmerTest.limit = 22404' (or larger)
## [or, globally, 'set emm_options(lmerTest.limit = 22404)' or larger];
## but be warned that this may result in large computation time and memory use.
# Summarize the results
summary(emm_results)
##  Question_Num emmean     SE  df asymp.LCL asymp.UCL
##             0  1.079 0.0945 Inf     0.893     1.264
##            10  1.071 0.0915 Inf     0.892     1.250
##            20  1.063 0.0885 Inf     0.890     1.236
##            30  1.055 0.0855 Inf     0.888     1.223
##            40  1.047 0.0825 Inf     0.886     1.209
##            50  1.039 0.0795 Inf     0.884     1.195
##            60  1.032 0.0765 Inf     0.882     1.182
##            70  1.024 0.0736 Inf     0.879     1.168
##            80  1.016 0.0707 Inf     0.877     1.154
##            90  1.008 0.0678 Inf     0.875     1.141
##           100  1.000 0.0649 Inf     0.873     1.127
##           110  0.992 0.0621 Inf     0.871     1.114
##           120  0.984 0.0593 Inf     0.868     1.101
##           130  0.977 0.0566 Inf     0.866     1.087
##           140  0.969 0.0539 Inf     0.863     1.074
##           150  0.961 0.0513 Inf     0.860     1.061
##           160  0.953 0.0487 Inf     0.857     1.048
##           170  0.945 0.0463 Inf     0.854     1.036
##           180  0.937 0.0439 Inf     0.851     1.023
##           190  0.929 0.0417 Inf     0.848     1.011
##           200  0.922 0.0395 Inf     0.844     0.999
##           210  0.914 0.0376 Inf     0.840     0.987
##           220  0.906 0.0358 Inf     0.836     0.976
##           230  0.898 0.0342 Inf     0.831     0.965
##           240  0.890 0.0329 Inf     0.826     0.955
##           250  0.882 0.0318 Inf     0.820     0.945
##           260  0.874 0.0311 Inf     0.813     0.935
##           270  0.866 0.0306 Inf     0.806     0.926
##           280  0.859 0.0305 Inf     0.799     0.918
##           290  0.851 0.0307 Inf     0.791     0.911
##           300  0.843 0.0313 Inf     0.782     0.904
##           310  0.835 0.0322 Inf     0.772     0.898
##           320  0.827 0.0333 Inf     0.762     0.893
##           330  0.819 0.0348 Inf     0.751     0.887
##           340  0.811 0.0364 Inf     0.740     0.883
##           350  0.804 0.0383 Inf     0.729     0.879
##           360  0.796 0.0403 Inf     0.717     0.875
##           370  0.788 0.0424 Inf     0.705     0.871
##           380  0.780 0.0447 Inf     0.692     0.868
## 
## Degrees-of-freedom method: asymptotic 
## Confidence level used: 0.95
# Plot the estimated marginal means
plot(emm_results)

# Create a plot of Reaction_Time vs. Question_Num with a regression line
ggplot(data, aes(x = Question_Num, y = Reaction_Time)) +
  geom_point(alpha = 0.3) +
  geom_smooth(method = "lm", se = TRUE, color = "blue") +
  labs(title = "Reaction Time vs. Question Number",
       x = "Question Number",
       y = "Reaction Time")
## `geom_smooth()` using formula = 'y ~ x'

2. Which condition is the hardest overall?

2a: rt - 1 + condition + (1 + condition | participant)

1 | Player_ID: Introduces a random intercept for each player, allowing baseline reaction times to vary. Phrase_Condition | Player_ID: Adds random slopes for the effect of Phrase_Condition on reaction time for each player, indicating that players may react differently to each phrase condition. Implications of Including Random Slopes: Individual Variability: Acknowledges that the effect of phrase conditions varies among players, leading to a more accurate model by accounting for interaction-like effects. Complexity: Increases the number of parameters to estimate, including a variance component for the random slopes.

model3 <- lmer(Reaction_Time ~ 1 + Phrase_Condition + (1 + Phrase_Condition | Player_ID), data = data)
## Warning in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
## Model failed to converge with max|grad| = 0.00768178 (tol = 0.002, component 1)
# Obtain and print a summary of the model to examine fixed and random effects
print(summary(model3))
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: Reaction_Time ~ 1 + Phrase_Condition + (1 + Phrase_Condition |  
##     Player_ID)
##    Data: data
## 
## REML criterion at convergence: 62721.8
## 
## Scaled residuals: 
##    Min     1Q Median     3Q    Max 
## -2.566 -0.342 -0.103  0.174 48.516 
## 
## Random effects:
##  Groups    Name              Variance Std.Dev. Corr             
##  Player_ID (Intercept)       0.008713 0.09334                   
##            Phrase_Condition2 0.031595 0.17775   0.29            
##            Phrase_Condition3 0.026379 0.16242   0.07  0.62      
##            Phrase_Condition4 0.098637 0.31407   0.40  0.20 -0.04
##  Residual                    0.950765 0.97507                   
## Number of obs: 22404, groups:  Player_ID, 38
## 
## Fixed effects:
##                   Estimate Std. Error       df t value Pr(>|t|)    
## (Intercept)        0.67601    0.02165 38.30934  31.227  < 2e-16 ***
## Phrase_Condition2  0.26567    0.03629 36.93804   7.322 1.07e-08 ***
## Phrase_Condition3  0.05920    0.03407 37.95185   1.738   0.0904 .  
## Phrase_Condition4  0.51368    0.05429 38.00528   9.462 1.55e-11 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation of Fixed Effects:
##             (Intr) Phr_C2 Phr_C3
## Phrs_Cndtn2 -0.143              
## Phrs_Cndtn3 -0.288  0.577       
## Phrs_Cndtn4  0.059  0.267  0.101
## optimizer (nloptwrap) convergence code: 0 (OK)
## Model failed to converge with max|grad| = 0.00768178 (tol = 0.002, component 1)
#Diagnostic plots
plot_residuals_diagnostics(model3)

## `geom_smooth()` using formula = 'y ~ x'

# Diagnostic checks - Simulating residuals to validate model assumptions
residuals_simulation <- simulateResiduals(fittedModel = model3, n = 500)
plot(residuals_simulation)

# Model diagnostics
check_model(model3)

# Check for influential cases potentially affecting the model
#influence_measures <- influence(model3)
#plot(influence_measures, which = "cook")

# If assumptions are not met, consider re-fitting the model with transformations or different specifications
# model_revised <- update(model, . ~ . + log(Reaction_Time))
# print(summary(model_revised))

# Model interpretation - examining fixed and random effects
#fixed_effects <- fixef(model3)
#random_effects <- ranef(model3)

# Printing fixed effects for interpretation
#print(fixed_effects)

# Printing random effects for interpretation
#print(random_effects)

# Conducting pairwise comparisons of phrase conditions using estimated marginal means
emm_results <- emmeans(model3, specs = ~ Phrase_Condition)
## Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
## To enable adjustments, add the argument 'pbkrtest.limit = 22404' (or larger)
## [or, globally, 'set emm_options(pbkrtest.limit = 22404)' or larger];
## but be warned that this may result in large computation time and memory use.
## Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
## To enable adjustments, add the argument 'lmerTest.limit = 22404' (or larger)
## [or, globally, 'set emm_options(lmerTest.limit = 22404)' or larger];
## but be warned that this may result in large computation time and memory use.
summary(emm_results)
##  Phrase_Condition emmean     SE  df asymp.LCL asymp.UCL
##  1                 0.676 0.0216 Inf     0.634     0.718
##  2                 0.942 0.0395 Inf     0.864     1.019
##  3                 0.735 0.0347 Inf     0.667     0.803
##  4                 1.190 0.0596 Inf     1.073     1.307
## 
## Degrees-of-freedom method: asymptotic 
## Confidence level used: 0.95
pairwise_comparisons <- emmeans(model3, specs = pairwise ~ Phrase_Condition)
## Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
## To enable adjustments, add the argument 'pbkrtest.limit = 22404' (or larger)
## [or, globally, 'set emm_options(pbkrtest.limit = 22404)' or larger];
## but be warned that this may result in large computation time and memory use.
## Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
## To enable adjustments, add the argument 'lmerTest.limit = 22404' (or larger)
## [or, globally, 'set emm_options(lmerTest.limit = 22404)' or larger];
## but be warned that this may result in large computation time and memory use.
summary(pairwise_comparisons)
## $emmeans
##  Phrase_Condition emmean     SE  df asymp.LCL asymp.UCL
##  1                 0.676 0.0216 Inf     0.634     0.718
##  2                 0.942 0.0395 Inf     0.864     1.019
##  3                 0.735 0.0347 Inf     0.667     0.803
##  4                 1.190 0.0596 Inf     1.073     1.307
## 
## Degrees-of-freedom method: asymptotic 
## Confidence level used: 0.95 
## 
## $contrasts
##  contrast                              estimate     SE  df z.ratio p.value
##  Phrase_Condition1 - Phrase_Condition2  -0.2657 0.0363 Inf  -7.322  <.0001
##  Phrase_Condition1 - Phrase_Condition3  -0.0592 0.0341 Inf  -1.738  0.3041
##  Phrase_Condition1 - Phrase_Condition4  -0.5137 0.0543 Inf  -9.462  <.0001
##  Phrase_Condition2 - Phrase_Condition3   0.2065 0.0324 Inf   6.367  <.0001
##  Phrase_Condition2 - Phrase_Condition4  -0.2480 0.0567 Inf  -4.376  0.0001
##  Phrase_Condition3 - Phrase_Condition4  -0.4545 0.0611 Inf  -7.439  <.0001
## 
## Degrees-of-freedom method: asymptotic 
## P value adjustment: tukey method for comparing a family of 4 estimates
plot(emm_results)

Overall Model Interpretation: This model shows how different phrase conditions affect reaction times, accounting for both average player effects and individual differences. Conditions 2 and 4 significantly increase reaction times, suggesting they are more challenging. This can guide game difficulty adjustments or targeted practice for players struggling with specific phrases. The random slopes indicate that these conditions impact players differently, supporting personalized gameplay or learning approaches.

Condition Effects:

Phrase_Condition2: Increases reaction time by 0.23230 seconds (p < 1.27e-10). Phrase_Condition3: Increases reaction time by 0.03446 seconds, not statistically significant (p = 0.106). Phrase_Condition4: Increases reaction time by 0.39736 seconds (p < 1.74e-14). Phrase_Condition4 is the hardest, with the largest significant effect, followed by Phrase_Condition2. Phrase_Condition3 does not significantly differ from the baseline.

# Create a boxplot
ggplot(data, aes(x = Phrase_Condition, y = Reaction_Time, fill = Phrase_Condition)) +
  geom_boxplot() +
  labs(title = "Reaction Times by Phrase Condition", x = "Phrase Condition", y = "Reaction Time (seconds)") +
  theme_minimal()

# Calculate average reaction times per player per condition
avg_reaction_times <- data %>%
  group_by(Player_ID, Phrase_Condition) %>%
  summarise(Avg_Reaction_Time = mean(Reaction_Time), .groups = 'drop')

# Interaction plot
ggplot(avg_reaction_times, aes(x = Phrase_Condition, y = Avg_Reaction_Time, group = Player_ID, color = Phrase_Condition)) +
  geom_line(alpha = 0.5) +  # Slightly transparent to see overlapping lines
  geom_point() +
  labs(title = "Interaction of Phrase Condition and Player on Reaction Times", x = "Phrase Condition", y = "Average Reaction Time (seconds)") +
  theme_minimal()

Jorge: ## 3. Does the version of the experiment cause different outcomes?

model8 <- lmer(Reaction_Time ~ 1 + Game_Version + (1 | Player_ID), data = data)

# Obtain and print a summary of the model to examine fixed and random effects
print(summary(model8))
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: Reaction_Time ~ 1 + Game_Version + (1 | Player_ID)
##    Data: data
## 
## REML criterion at convergence: 64210.2
## 
## Scaled residuals: 
##    Min     1Q Median     3Q    Max 
## -1.468 -0.389 -0.111  0.199 46.837 
## 
## Random effects:
##  Groups    Name        Variance Std.Dev.
##  Player_ID (Intercept) 0.04381  0.2093  
##  Residual              1.02245  1.0112  
## Number of obs: 22404, groups:  Player_ID, 38
## 
## Fixed effects:
##               Estimate Std. Error       df t value Pr(>|t|)    
## (Intercept)    0.85459    0.07143 34.55240  11.964  7.7e-14 ***
## Game_Version2  0.17590    0.10093 34.44373   1.743   0.0903 .  
## Game_Version3  0.15994    0.09826 34.27187   1.628   0.1127    
## Game_Version4  0.06567    0.09822 34.22056   0.669   0.5082    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation of Fixed Effects:
##             (Intr) Gm_Vr2 Gm_Vr3
## Game_Versn2 -0.708              
## Game_Versn3 -0.727  0.514       
## Game_Versn4 -0.727  0.515  0.529
#Diagnostic plots
plot_residuals_diagnostics(model8)

## `geom_smooth()` using formula = 'y ~ x'

# Diagnostic checks - Simulating residuals to validate model assumptions
residuals_simulation <- simulateResiduals(fittedModel = model8, n = 500)
plot(residuals_simulation)

# Model diagnostics
check_model(model8)

# Check for influential cases potentially affecting the model
#influence_measures <- influence(model4)
#plot(influence_measures, which = "cook")

# If assumptions are not met, consider re-fitting the model with transformations or different specifications
# model_revised <- update(model, . ~ . + log(Reaction_Time))
# print(summary(model_revised))

# Model interpretation - examining fixed and random effects
fixed_effects <- fixef(model8)
random_effects <- ranef(model8)

How does the effect of condition vary depending on the version?

4b: rt ~ 1 + condition * version + (1 + condition | participant)

Main Effects:

Phrase_Condition: Measures how different phrase conditions affect reaction times compared to the baseline condition. Game_Version: Evaluates the impact of different game versions on reaction times relative to the baseline version. Interaction Effect (Phrase_Condition * Game_Version):

Assesses if the effect of a phrase condition on reaction times changes across game versions. Each interaction coefficient shows how the combined effect of a phrase condition and game version differs from their individual effects. Random Effects:

(1 | Player_ID): Models random intercepts for each player, indicating variability in baseline reaction times not explained by phrase conditions or game versions. Each player has an adjusted baseline reaction time. Interpretation:

Intercept: Represents the average reaction time for the baseline phrase condition and game version. Main Effects: Coefficients show how reaction times change with each phrase condition and game version, assuming no interaction. Interaction Effects: Significant interaction terms indicate that the effect of a phrase condition on reaction time is modified by the game version, suggesting areas for tailored game design to optimize learning or user experience.

#This doesnt work:
#model4 <- lmer(Reaction_Time ~ 1 + Phrase_Condition * Game_Version + (1 + Phrase_Condition | Player_ID), data = data)

model4 <- lmer(Reaction_Time ~ 1 + Phrase_Condition * Game_Version + (1 | Player_ID), data = data)

# Obtain and print a summary of the model to examine fixed and random effects
print(summary(model4))
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: Reaction_Time ~ 1 + Phrase_Condition * Game_Version + (1 | Player_ID)
##    Data: data
## 
## REML criterion at convergence: 63270.8
## 
## Scaled residuals: 
##    Min     1Q Median     3Q    Max 
## -1.755 -0.357 -0.110  0.181 47.728 
## 
## Random effects:
##  Groups    Name        Variance Std.Dev.
##  Player_ID (Intercept) 0.04723  0.2173  
##  Residual              0.97828  0.9891  
## Number of obs: 22404, groups:  Player_ID, 38
## 
## Fixed effects:
##                                   Estimate Std. Error         df t value
## (Intercept)                      7.178e-01  7.988e-02  4.692e+01   8.987
## Phrase_Condition2                1.388e-01  4.755e-02  2.235e+04   2.919
## Phrase_Condition3               -8.641e-02  4.672e-02  2.236e+04  -1.849
## Phrase_Condition4                3.243e-01  4.122e-02  2.237e+04   7.867
## Game_Version2                   -3.493e-02  1.124e-01  4.596e+01  -0.311
## Game_Version3                   -8.644e-02  1.100e-01  4.673e+01  -0.786
## Game_Version4                   -3.894e-02  1.093e-01  4.553e+01  -0.356
## Phrase_Condition2:Game_Version2  3.150e-01  6.539e-02  2.236e+04   4.817
## Phrase_Condition3:Game_Version2  2.283e-01  6.441e-02  2.236e+04   3.545
## Phrase_Condition4:Game_Version2  2.647e-01  5.685e-02  2.237e+04   4.657
## Phrase_Condition2:Game_Version3  2.159e-01  6.514e-02  2.235e+04   3.314
## Phrase_Condition3:Game_Version3  3.713e-01  6.309e-02  2.236e+04   5.885
## Phrase_Condition4:Game_Version3  2.459e-01  5.515e-02  2.237e+04   4.459
## Phrase_Condition2:Game_Version4 -6.528e-03  6.312e-02  2.236e+04  -0.103
## Phrase_Condition3:Game_Version4  7.052e-02  6.188e-02  2.236e+04   1.140
## Phrase_Condition4:Game_Version4  1.493e-01  5.333e-02  2.237e+04   2.799
##                                 Pr(>|t|)    
## (Intercept)                     9.09e-12 ***
## Phrase_Condition2               0.003512 ** 
## Phrase_Condition3               0.064404 .  
## Phrase_Condition4               3.80e-15 ***
## Game_Version2                   0.757347    
## Game_Version3                   0.435910    
## Game_Version4                   0.723223    
## Phrase_Condition2:Game_Version2 1.47e-06 ***
## Phrase_Condition3:Game_Version2 0.000394 ***
## Phrase_Condition4:Game_Version2 3.23e-06 ***
## Phrase_Condition2:Game_Version3 0.000921 ***
## Phrase_Condition3:Game_Version3 4.03e-09 ***
## Phrase_Condition4:Game_Version3 8.28e-06 ***
## Phrase_Condition2:Game_Version4 0.917623    
## Phrase_Condition3:Game_Version4 0.254420    
## Phrase_Condition4:Game_Version4 0.005133 ** 
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation matrix not shown by default, as p = 16 > 12.
## Use print(summary(model4), correlation=TRUE)  or
##     vcov(summary(model4))        if you need it
#Diagnostic plots
plot_residuals_diagnostics(model4)

## `geom_smooth()` using formula = 'y ~ x'

# Diagnostic checks - Simulating residuals to validate model assumptions
residuals_simulation <- simulateResiduals(fittedModel = model4, n = 500)
plot(residuals_simulation)

# Model diagnostics
check_model(model4)

# Check for influential cases potentially affecting the model
#influence_measures <- influence(model4)
#plot(influence_measures, which = "cook")

# If assumptions are not met, consider re-fitting the model with transformations or different specifications
# model_revised <- update(model, . ~ . + log(Reaction_Time))
# print(summary(model_revised))

# Model interpretation - examining fixed and random effects
fixed_effects <- fixef(model4)
random_effects <- ranef(model4)

# Printing fixed effects for interpretation
print(fixed_effects)
##                     (Intercept)               Phrase_Condition2 
##                     0.717835648                     0.138799455 
##               Phrase_Condition3               Phrase_Condition4 
##                    -0.086411190                     0.324269247 
##                   Game_Version2                   Game_Version3 
##                    -0.034929318                    -0.086438000 
##                   Game_Version4 Phrase_Condition2:Game_Version2 
##                    -0.038941112                     0.314959990 
## Phrase_Condition3:Game_Version2 Phrase_Condition4:Game_Version2 
##                     0.228307728                     0.264739313 
## Phrase_Condition2:Game_Version3 Phrase_Condition3:Game_Version3 
##                     0.215875147                     0.371271465 
## Phrase_Condition4:Game_Version3 Phrase_Condition2:Game_Version4 
##                     0.245890992                    -0.006528468 
## Phrase_Condition3:Game_Version4 Phrase_Condition4:Game_Version4 
##                     0.070520852                     0.149258028
# Printing random effects for interpretation
print(random_effects)
## $Player_ID
##     (Intercept)
## 1  -0.193400990
## 2  -0.172907500
## 3  -0.139512409
## 4   0.054738806
## 5   0.097357400
## 6   0.007593549
## 7   0.169121999
## 8  -0.015378239
## 9   0.192387383
## 10 -0.040921626
## 11 -0.119072729
## 12  0.277477969
## 13 -0.138723069
## 14 -0.088199447
## 15  0.298225096
## 16  0.149117761
## 17 -0.379697071
## 18  0.041793116
## 19  0.091510015
## 20 -0.109400826
## 21 -0.038363901
## 22  0.014469477
## 23 -0.279909985
## 24  0.106276801
## 25  0.341888139
## 26  0.134232377
## 27 -0.152810008
## 28 -0.107892089
## 29  0.152628995
## 30 -0.098880689
## 31  0.083349923
## 32  0.021809041
## 33  0.653097901
## 34 -0.288705849
## 35  0.136633493
## 36 -0.186142609
## 37 -0.300622841
## 38 -0.173167365
## 
## with conditional variances for "Player_ID"
# Estimate marginal means for all combinations
emm1 <- emmeans(model4, specs = ~ Phrase_Condition * Game_Version)
## Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
## To enable adjustments, add the argument 'pbkrtest.limit = 22404' (or larger)
## [or, globally, 'set emm_options(pbkrtest.limit = 22404)' or larger];
## but be warned that this may result in large computation time and memory use.
## Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
## To enable adjustments, add the argument 'lmerTest.limit = 22404' (or larger)
## [or, globally, 'set emm_options(lmerTest.limit = 22404)' or larger];
## but be warned that this may result in large computation time and memory use.
summary(emm1)
##  Phrase_Condition Game_Version emmean     SE  df asymp.LCL asymp.UCL
##  1                1             0.718 0.0799 Inf     0.561     0.874
##  2                1             0.857 0.0799 Inf     0.700     1.013
##  3                1             0.631 0.0794 Inf     0.476     0.787
##  4                1             1.042 0.0763 Inf     0.893     1.192
##  1                2             0.683 0.0791 Inf     0.528     0.838
##  2                2             1.137 0.0792 Inf     0.981     1.292
##  3                2             0.825 0.0789 Inf     0.670     0.979
##  4                2             1.272 0.0761 Inf     1.123     1.421
##  1                3             0.631 0.0756 Inf     0.483     0.780
##  2                3             0.986 0.0756 Inf     0.838     1.134
##  3                3             0.916 0.0743 Inf     0.771     1.062
##  4                3             1.202 0.0712 Inf     1.062     1.341
##  1                4             0.679 0.0746 Inf     0.533     0.825
##  2                4             0.811 0.0749 Inf     0.664     0.958
##  3                4             0.663 0.0744 Inf     0.517     0.809
##  4                4             1.152 0.0710 Inf     1.013     1.292
## 
## Degrees-of-freedom method: asymptotic 
## Confidence level used: 0.95
# Pairwise comparisons across phrase conditions within each game version
pairs(emm1, simple = "each")
## $`simple contrasts for Phrase_Condition`
## Game_Version = 1:
##  contrast                              estimate     SE  df z.ratio p.value
##  Phrase_Condition1 - Phrase_Condition2  -0.1388 0.0475 Inf  -2.919  0.0184
##  Phrase_Condition1 - Phrase_Condition3   0.0864 0.0467 Inf   1.849  0.2502
##  Phrase_Condition1 - Phrase_Condition4  -0.3243 0.0412 Inf  -7.867  <.0001
##  Phrase_Condition2 - Phrase_Condition3   0.2252 0.0467 Inf   4.824  <.0001
##  Phrase_Condition2 - Phrase_Condition4  -0.1855 0.0412 Inf  -4.504  <.0001
##  Phrase_Condition3 - Phrase_Condition4  -0.4107 0.0400 Inf -10.260  <.0001
## 
## Game_Version = 2:
##  contrast                              estimate     SE  df z.ratio p.value
##  Phrase_Condition1 - Phrase_Condition2  -0.4538 0.0449 Inf -10.108  <.0001
##  Phrase_Condition1 - Phrase_Condition3  -0.1419 0.0443 Inf  -3.201  0.0075
##  Phrase_Condition1 - Phrase_Condition4  -0.5890 0.0392 Inf -15.044  <.0001
##  Phrase_Condition2 - Phrase_Condition3   0.3119 0.0446 Inf   6.992  <.0001
##  Phrase_Condition2 - Phrase_Condition4  -0.1352 0.0396 Inf  -3.412  0.0036
##  Phrase_Condition3 - Phrase_Condition4  -0.4471 0.0389 Inf -11.494  <.0001
## 
## Game_Version = 3:
##  contrast                              estimate     SE  df z.ratio p.value
##  Phrase_Condition1 - Phrase_Condition2  -0.3547 0.0445 Inf  -7.966  <.0001
##  Phrase_Condition1 - Phrase_Condition3  -0.2849 0.0424 Inf  -6.720  <.0001
##  Phrase_Condition1 - Phrase_Condition4  -0.5702 0.0366 Inf -15.562  <.0001
##  Phrase_Condition2 - Phrase_Condition3   0.0698 0.0423 Inf   1.650  0.3505
##  Phrase_Condition2 - Phrase_Condition4  -0.2155 0.0365 Inf  -5.897  <.0001
##  Phrase_Condition3 - Phrase_Condition4  -0.2853 0.0337 Inf  -8.460  <.0001
## 
## Game_Version = 4:
##  contrast                              estimate     SE  df z.ratio p.value
##  Phrase_Condition1 - Phrase_Condition2  -0.1323 0.0415 Inf  -3.186  0.0079
##  Phrase_Condition1 - Phrase_Condition3   0.0159 0.0406 Inf   0.392  0.9796
##  Phrase_Condition1 - Phrase_Condition4  -0.4735 0.0338 Inf -13.994  <.0001
##  Phrase_Condition2 - Phrase_Condition3   0.1482 0.0412 Inf   3.598  0.0018
##  Phrase_Condition2 - Phrase_Condition4  -0.3413 0.0347 Inf  -9.845  <.0001
##  Phrase_Condition3 - Phrase_Condition4  -0.4894 0.0330 Inf -14.833  <.0001
## 
## Degrees-of-freedom method: asymptotic 
## P value adjustment: tukey method for comparing a family of 4 estimates 
## 
## $`simple contrasts for Game_Version`
## Phrase_Condition = 1:
##  contrast                      estimate    SE  df z.ratio p.value
##  Game_Version1 - Game_Version2  0.03493 0.112 Inf   0.311  0.9896
##  Game_Version1 - Game_Version3  0.08644 0.110 Inf   0.786  0.8609
##  Game_Version1 - Game_Version4  0.03894 0.109 Inf   0.356  0.9845
##  Game_Version2 - Game_Version3  0.05151 0.109 Inf   0.471  0.9655
##  Game_Version2 - Game_Version4  0.00401 0.109 Inf   0.037  1.0000
##  Game_Version3 - Game_Version4 -0.04750 0.106 Inf  -0.447  0.9702
## 
## Phrase_Condition = 2:
##  contrast                      estimate    SE  df z.ratio p.value
##  Game_Version1 - Game_Version2 -0.28003 0.112 Inf  -2.489  0.0615
##  Game_Version1 - Game_Version3 -0.12944 0.110 Inf  -1.177  0.6412
##  Game_Version1 - Game_Version4  0.04547 0.109 Inf   0.415  0.9759
##  Game_Version2 - Game_Version3  0.15059 0.109 Inf   1.375  0.5149
##  Game_Version2 - Game_Version4  0.32550 0.109 Inf   2.985  0.0150
##  Game_Version3 - Game_Version4  0.17491 0.106 Inf   1.644  0.3540
## 
## Phrase_Condition = 3:
##  contrast                      estimate    SE  df z.ratio p.value
##  Game_Version1 - Game_Version2 -0.19338 0.112 Inf  -1.728  0.3090
##  Game_Version1 - Game_Version3 -0.28483 0.109 Inf  -2.619  0.0437
##  Game_Version1 - Game_Version4 -0.03158 0.109 Inf  -0.290  0.9915
##  Game_Version2 - Game_Version3 -0.09146 0.108 Inf  -0.844  0.8335
##  Game_Version2 - Game_Version4  0.16180 0.108 Inf   1.492  0.4424
##  Game_Version3 - Game_Version4  0.25325 0.105 Inf   2.408  0.0756
## 
## Phrase_Condition = 4:
##  contrast                      estimate    SE  df z.ratio p.value
##  Game_Version1 - Game_Version2 -0.22981 0.108 Inf  -2.134  0.1425
##  Game_Version1 - Game_Version3 -0.15945 0.104 Inf  -1.528  0.4204
##  Game_Version1 - Game_Version4 -0.11032 0.104 Inf  -1.059  0.7148
##  Game_Version2 - Game_Version3  0.07036 0.104 Inf   0.675  0.9065
##  Game_Version2 - Game_Version4  0.11949 0.104 Inf   1.148  0.6597
##  Game_Version3 - Game_Version4  0.04914 0.101 Inf   0.489  0.9617
## 
## Degrees-of-freedom method: asymptotic 
## P value adjustment: tukey method for comparing a family of 4 estimates
plot(emm1)

- Phrase_Condition4 shows a substantial increase in reaction times, suggesting it is more challenging than the baseline condition. - Interactions such as Phrase_Condition2:Game_Version2 and Phrase_Condition3:Game_Version3 are significantly affecting reaction times, indicating that the combination of these conditions and versions exacerbates or alters the impact seen with the conditions alone. - Non-significant main effects for Game_Version (2, 3, and 4) suggest that changes in game versions alone do not significantly alter reaction times, but their interactions with phrase conditions do. - Practical Significance: The model indicates that while some game versions and phrase conditions significantly affect reaction times, others do not. This can inform adjustments in game design or targeted interventions for training or improvement purposes. - Model Fit and Quality: The REML criterion and scaled residuals indicate the model fits the data well. Scaled residuals spread mostly symmetrically around zero, suggesting adequate model fit without apparent bias.

5. Does the rate of improvement in each condition depend on the version?

5a: rt ~ 1 + time + condition * version + (1 + time + condition | participant)

1 (Intercept): Represents the baseline reaction time for the first question and the baseline categories of Phrase_Condition and Game_Version.

Question_Num: Continuous variable representing the sequential number of questions. A negative coefficient indicates improved reaction time (i.e., decreasing reaction time) as the game progresses.

Phrase_Condition * Game_Version: Examines if the effect of phrase conditions on reaction time is modified by the game version, identifying combinations where game version influences the difficulty of phrase conditions.

(1 | Player_ID): Random intercepts for players account for individual differences in baseline reaction times, acknowledging that some players generally respond faster or slower than others, independent of condition or version.

model5 <- lmer(Reaction_Time ~ 1 + Question_Num + Phrase_Condition * Game_Version + (1 | Player_ID), data = data)

# Obtain and print a summary of the model to examine fixed and random effects
print(summary(model5))
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: Reaction_Time ~ 1 + Question_Num + Phrase_Condition * Game_Version +  
##     (1 | Player_ID)
##    Data: data
## 
## REML criterion at convergence: 63274.3
## 
## Scaled residuals: 
##    Min     1Q Median     3Q    Max 
## -1.766 -0.358 -0.108  0.184 47.738 
## 
## Random effects:
##  Groups    Name        Variance Std.Dev.
##  Player_ID (Intercept) 0.0474   0.2177  
##  Residual              0.9777   0.9888  
## Number of obs: 22404, groups:  Player_ID, 38
## 
## Fixed effects:
##                                   Estimate Std. Error         df t value
## (Intercept)                      7.535e-01  8.058e-02  4.824e+01   9.352
## Question_Num                    -2.867e-04  7.781e-05  2.236e+04  -3.685
## Phrase_Condition2                1.457e-01  4.757e-02  2.235e+04   3.064
## Phrase_Condition3               -5.557e-02  4.745e-02  2.236e+04  -1.171
## Phrase_Condition4                3.565e-01  4.213e-02  2.237e+04   8.463
## Game_Version2                    3.184e-03  1.130e-01  4.668e+01   0.028
## Game_Version3                   -5.439e-02  1.105e-01  4.726e+01  -0.492
## Game_Version4                   -3.190e-02  1.095e-01  4.551e+01  -0.291
## Phrase_Condition2:Game_Version2  2.708e-01  6.646e-02  2.235e+04   4.075
## Phrase_Condition3:Game_Version2  1.652e-01  6.662e-02  2.236e+04   2.480
## Phrase_Condition4:Game_Version2  2.166e-01  5.832e-02  2.237e+04   3.714
## Phrase_Condition2:Game_Version3  2.151e-01  6.512e-02  2.235e+04   3.304
## Phrase_Condition3:Game_Version3  3.068e-01  6.545e-02  2.236e+04   4.688
## Phrase_Condition4:Game_Version3  1.751e-01  5.838e-02  2.237e+04   2.999
## Phrase_Condition2:Game_Version4  1.060e-02  6.327e-02  2.236e+04   0.168
## Phrase_Condition3:Game_Version4  7.043e-02  6.186e-02  2.236e+04   1.139
## Phrase_Condition4:Game_Version4  9.266e-02  5.548e-02  2.237e+04   1.670
##                                 Pr(>|t|)    
## (Intercept)                     2.06e-12 ***
## Question_Num                    0.000229 ***
## Phrase_Condition2               0.002189 ** 
## Phrase_Condition3               0.241561    
## Phrase_Condition4                < 2e-16 ***
## Game_Version2                   0.977645    
## Game_Version3                   0.624813    
## Game_Version4                   0.772057    
## Phrase_Condition2:Game_Version2 4.62e-05 ***
## Phrase_Condition3:Game_Version2 0.013153 *  
## Phrase_Condition4:Game_Version2 0.000204 ***
## Phrase_Condition2:Game_Version3 0.000955 ***
## Phrase_Condition3:Game_Version3 2.77e-06 ***
## Phrase_Condition4:Game_Version3 0.002709 ** 
## Phrase_Condition2:Game_Version4 0.866902    
## Phrase_Condition3:Game_Version4 0.254880    
## Phrase_Condition4:Game_Version4 0.094913 .  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation matrix not shown by default, as p = 17 > 12.
## Use print(summary(model5), correlation=TRUE)  or
##     vcov(summary(model5))        if you need it
#Diagnostic plots
plot_residuals_diagnostics(model5)

## `geom_smooth()` using formula = 'y ~ x'

# Diagnostic checks - Simulating residuals to validate model assumptions
residuals_simulation <- simulateResiduals(fittedModel = model5, n = 500)
plot(residuals_simulation)

# Model diagnostics
check_model(model5)

# Check for influential cases potentially affecting the model
#influence_measures <- influence(model5)
#plot(influence_measures, which = "cook")

# If assumptions are not met, consider re-fitting the model with transformations or different specifications
# model_revised <- update(model, . ~ . + log(Reaction_Time))
# print(summary(model_revised))

# Model interpretation - examining fixed and random effects
fixed_effects <- fixef(model5)
random_effects <- ranef(model5)

# Printing fixed effects for interpretation
print(fixed_effects)
##                     (Intercept)                    Question_Num 
##                    0.7535226350                   -0.0002867384 
##               Phrase_Condition2               Phrase_Condition3 
##                    0.1457410256                   -0.0555730125 
##               Phrase_Condition4                   Game_Version2 
##                    0.3565391405                    0.0031839946 
##                   Game_Version3                   Game_Version4 
##                   -0.0543915089                   -0.0318970820 
## Phrase_Condition2:Game_Version2 Phrase_Condition3:Game_Version2 
##                    0.2708267333                    0.1652151615 
## Phrase_Condition4:Game_Version2 Phrase_Condition2:Game_Version3 
##                    0.2166081933                    0.2151443866 
## Phrase_Condition3:Game_Version3 Phrase_Condition4:Game_Version3 
##                    0.3068451377                    0.1751056415 
## Phrase_Condition2:Game_Version4 Phrase_Condition3:Game_Version4 
##                    0.0106042643                    0.0704325692 
## Phrase_Condition4:Game_Version4 
##                    0.0926615960
# Printing random effects for interpretation
print(random_effects)
## $Player_ID
##     (Intercept)
## 1  -0.192754145
## 2  -0.174395194
## 3  -0.138741453
## 4   0.055547188
## 5   0.097623412
## 6   0.006281094
## 7   0.169758781
## 8  -0.015250655
## 9   0.191930974
## 10 -0.039913031
## 11 -0.118079192
## 12  0.277690794
## 13 -0.137919769
## 14 -0.088200433
## 15  0.298405759
## 16  0.146284999
## 17 -0.382323726
## 18  0.044054599
## 19  0.087427025
## 20 -0.110594339
## 21 -0.039061676
## 22  0.018831239
## 23 -0.276981734
## 24  0.103730625
## 25  0.340950792
## 26  0.134942965
## 27 -0.151602715
## 28 -0.107642183
## 29  0.155285600
## 30 -0.099797419
## 31  0.082875172
## 32  0.020498780
## 33  0.653603944
## 34 -0.290253687
## 35  0.143599566
## 36 -0.187536411
## 37 -0.306304699
## 38 -0.171970848
## 
## with conditional variances for "Player_ID"
# Estimate marginal means at various levels of Question_Num
emm2 <- emmeans(model5, specs = ~ Phrase_Condition * Game_Version, at = list(Question_Num = c(1, 10, 20)))
## Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
## To enable adjustments, add the argument 'pbkrtest.limit = 22404' (or larger)
## [or, globally, 'set emm_options(pbkrtest.limit = 22404)' or larger];
## but be warned that this may result in large computation time and memory use.
## Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
## To enable adjustments, add the argument 'lmerTest.limit = 22404' (or larger)
## [or, globally, 'set emm_options(lmerTest.limit = 22404)' or larger];
## but be warned that this may result in large computation time and memory use.
summary(emm2)
##  Phrase_Condition Game_Version emmean     SE  df asymp.LCL asymp.UCL
##  1                1             0.751 0.0805 Inf     0.593     0.908
##  2                1             0.896 0.0807 Inf     0.738     1.054
##  3                1             0.695 0.0813 Inf     0.536     0.854
##  4                1             1.107 0.0784 Inf     0.953     1.261
##  1                2             0.754 0.0815 Inf     0.594     0.913
##  2                2             1.170 0.0799 Inf     1.014     1.327
##  3                2             0.863 0.0797 Inf     0.707     1.020
##  4                2             1.327 0.0776 Inf     1.175     1.479
##  1                3             0.696 0.0777 Inf     0.544     0.849
##  2                3             1.057 0.0781 Inf     0.904     1.210
##  3                3             0.947 0.0749 Inf     0.801     1.094
##  4                3             1.228 0.0717 Inf     1.087     1.368
##  1                4             0.719 0.0755 Inf     0.571     0.867
##  2                4             0.875 0.0770 Inf     0.724     1.026
##  3                4             0.734 0.0770 Inf     0.583     0.884
##  4                4             1.168 0.0713 Inf     1.028     1.308
## 
## Results are averaged over the levels of: Question_Num 
## Degrees-of-freedom method: asymptotic 
## Confidence level used: 0.95
# Trend analysis
trend_analysis <- emtrends(model5, specs = ~ Phrase_Condition * Game_Version, var = "Question_Num")
## Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
## To enable adjustments, add the argument 'pbkrtest.limit = 22404' (or larger)
## [or, globally, 'set emm_options(pbkrtest.limit = 22404)' or larger];
## but be warned that this may result in large computation time and memory use.
## Note: D.f. calculations have been disabled because the number of observations exceeds 3000.
## To enable adjustments, add the argument 'lmerTest.limit = 22404' (or larger)
## [or, globally, 'set emm_options(lmerTest.limit = 22404)' or larger];
## but be warned that this may result in large computation time and memory use.
summary(trend_analysis)
##  Phrase_Condition Game_Version Question_Num.trend       SE  df asymp.LCL
##  1                1                     -0.000287 7.78e-05 Inf -0.000439
##  2                1                     -0.000287 7.78e-05 Inf -0.000439
##  3                1                     -0.000287 7.78e-05 Inf -0.000439
##  4                1                     -0.000287 7.78e-05 Inf -0.000439
##  1                2                     -0.000287 7.78e-05 Inf -0.000439
##  2                2                     -0.000287 7.78e-05 Inf -0.000439
##  3                2                     -0.000287 7.78e-05 Inf -0.000439
##  4                2                     -0.000287 7.78e-05 Inf -0.000439
##  1                3                     -0.000287 7.78e-05 Inf -0.000439
##  2                3                     -0.000287 7.78e-05 Inf -0.000439
##  3                3                     -0.000287 7.78e-05 Inf -0.000439
##  4                3                     -0.000287 7.78e-05 Inf -0.000439
##  1                4                     -0.000287 7.78e-05 Inf -0.000439
##  2                4                     -0.000287 7.78e-05 Inf -0.000439
##  3                4                     -0.000287 7.78e-05 Inf -0.000439
##  4                4                     -0.000287 7.78e-05 Inf -0.000439
##  asymp.UCL
##  -0.000134
##  -0.000134
##  -0.000134
##  -0.000134
##  -0.000134
##  -0.000134
##  -0.000134
##  -0.000134
##  -0.000134
##  -0.000134
##  -0.000134
##  -0.000134
##  -0.000134
##  -0.000134
##  -0.000134
##  -0.000134
## 
## Degrees-of-freedom method: asymptotic 
## Confidence level used: 0.95
plot(emm2)

Model Overview

Formula: Reaction time ~ question number, phrase conditions, game versions, and their interactions, with a random intercept for each player. Data: 21,488 observations from 38 unique players. Random Effects

Player Variability: Variance = 0.01431 (Std.Dev. = 0.1196), indicating moderate variation. Residual Variability: Variance = 0.13911 (Std.Dev. = 0.3730), suggesting much of the variation remains unexplained. Fixed Effects

Intercept: Baseline reaction time is 0.6381 seconds. Question_Num: Positive but very small coefficient (1.384e-04), indicating minimal change over time. Phrase_Condition: Phrase_Condition4 significantly increases reaction times; Phrase_Condition2 increases slightly; Phrase_Condition3 shows a minor decrease. Game_Version: Versions 2, 3, and 4 do not significantly affect reaction times. Interaction Effects

Significant Positive Interactions: Certain combinations (e.g., Phrase_Condition3 and Game_Version3, Phrase_Condition2 and Game_Version2) increase reaction times more than individual effects. Non-significant/Negative Interactions: Some combinations do not significantly impact or slightly decrease reaction times (e.g., Phrase_Condition4). Interpretation and Implications

Interactions Matter: Highlights the importance of considering interactions between game design elements. Minimal Overall Time Trend: Very small positive coefficient for Question_Num suggests minimal change in reaction times, indicating little learning or fatigue effects. Model Fit and Residuals: Significant random effects variance indicates the model captures some complexity, but individual differences and unmodeled factors still contribute to variability in reaction times.

Let’s compare different models

# Fit the models as described
model6 <- lmer(Reaction_Time ~ 1 + Phrase_Condition * Game_Version + (1 | Player_ID), data = data)

# Model with random intercept and random slope for the interaction term
model7 <- lmer(Reaction_Time ~ 1 + Phrase_Condition * Game_Version + (1 + Phrase_Condition | Player_ID), data = data)
## boundary (singular) fit: see help('isSingular')
## Warning: Model failed to converge with 1 negative eigenvalue: -5.2e+01
# Model with random intercept and random slope for the interaction term
#model8 <- lmer(Reaction_Time ~ 1 + Phrase_Condition * Game_Version + 
#               (1 | Player_ID) + (0 + Phrase_Condition * Game_Version | Player_ID), data = data)

anova(model6, model7)
## refitting model(s) with ML (instead of REML)
## Data: data
## Models:
## model6: Reaction_Time ~ 1 + Phrase_Condition * Game_Version + (1 | Player_ID)
## model7: Reaction_Time ~ 1 + Phrase_Condition * Game_Version + (1 + Phrase_Condition | Player_ID)
##        npar   AIC   BIC logLik deviance  Chisq Df Pr(>Chisq)    
## model6   18 63235 63379 -31599    63199                         
## model7   27 62720 62936 -31333    62666 533.22  9  < 2.2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Interpretation 6: This model tells you how Phrase_Condition and Game_Version interact to influence reaction time, considering the average differences among players.

Interpretation 7: This model tells you how Phrase_Condition and Game_Version interact to influence reaction time, and also how the effect of Phrase_Condition on reaction time varies among players.

Interpretation 8: This model tells you how Phrase_Condition and Game_Version interact to influence reaction time, and how this interaction effect varies among players.

Model 6 vs. Model 7:

Model 6: Simpler, assumes the effect of Phrase_Condition is consistent across players. Model 7: More complex, allows the effect of Phrase_Condition to vary by player, providing a more individualized understanding of how Phrase_Condition impacts reaction time.

  1. Model Fit:
    • AIC and BIC: Both AIC and BIC are lower for Model 7 compared to Model 6, indicating that Model 7 provides a better fit to the data despite having more parameters. Lower values of AIC and BIC suggest a model with a better balance of goodness-of-fit and complexity.
    • Log-Likelihood: Model 7 has a higher log-likelihood (less negative) compared to Model 6, which also indicates a better fit.
  2. Deviance:
    • The deviance is lower for Model 7 (17477) compared to Model 6 (18753), indicating that Model 7 explains more of the variability in Reaction_Time.
  3. Chi-Square Test:
    • The Chi-Square test for the difference in deviance between the models is highly significant (Chi-Square = 1275.9, p < 2.2e-16). This significant result indicates that Model 7 provides a significantly better fit to the data than Model 6.
    • The degrees of freedom for the test is 9, corresponding to the difference in the number of parameters between the two models (27 - 18).

What Model 7 Tells You:

  • Improved Fit: Model 7, with random slopes for Phrase_Condition by Player_ID, fits the data significantly better than Model 6. This suggests that allowing the effect of Phrase_Condition to vary across players captures important variability in Reaction_Time.
  • Individual Differences: Model 7 accounts for individual differences in how Phrase_Condition affects reaction time, which is not captured in Model 6.

Conclusion:

Model 7 is a better choice for your data as it provides a significantly improved fit by allowing for random slopes for Phrase_Condition. This model acknowledges that the effect of Phrase_Condition on Reaction_Time varies among players, capturing individual differences that Model 6 does not.

Reformat the data to answer each Accuracy question

# Dataframe for learning (Accuracy over Time)
data_learning <- data %>%
  group_by(Player_ID, Question_Num) %>%
  summarise(Average_Accuracy = mean(Answer), .groups = 'drop')

# Dataframe for hardest condition
data_condition <- data %>%
  group_by(Player_ID, Phrase_Condition) %>%
  summarise(Average_Accuracy = mean(Answer), .groups = 'drop')

# Dataframe for the impact of different versions
data_version <- data %>%
  group_by(Player_ID, Game_Version) %>%
  summarise(Average_Accuracy = mean(Answer), .groups = 'drop')

# Dataframe for rate of improvement depending on the version and condition
data_improvement <- data %>%
  group_by(Player_ID, Question_Num, Phrase_Condition, Game_Version) %>%
  summarise(Average_Accuracy = mean(Answer), .groups = 'drop')

What is the average accuracy marginalizing over everything?

Ob: ac ~ 1 + (1 | participant)

# Dataframe for overall average accuracy per participant
data_ac <- data %>%
  group_by(Player_ID) %>%
  summarise(Average_Accuracy = mean(Answer), .groups = 'drop')

# Fit the model
model2 <- lm(Average_Accuracy ~ 1, data = data_ac)

# Obtain and print a summary of the model to examine fixed and random effects
print(summary(model2))
## 
## Call:
## lm(formula = Average_Accuracy ~ 1, data = data_ac)
## 
## Residuals:
##       Min        1Q    Median        3Q       Max 
## -0.114045 -0.035124  0.009193  0.035259  0.089426 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)   0.8342     0.0086      97   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.05302 on 37 degrees of freedom
#Diagnostic plots
#plot_residuals_diagnostics(model2)
plot(model2)

## hat values (leverages) are all = 0.02631579
##  and there are no factor predictors; no plot no. 5

# Diagnostic checks - Simulating residuals to validate model assumptions
residuals_simulation <- simulateResiduals(fittedModel = model2, n = 500)
plot(residuals_simulation)

# Model diagnostics
check_model(model2)

# If assumptions are not met, consider re-fitting the model with transformations or different specifications
# model_revised <- update(model, . ~ . + log(Reaction_Time))
# print(summary(model_revised))

How does the effect of condition vary depending on the version?

4b: ac - 1 + condition * version + (1 + condition | participant)

# Dataframe for condition and version interaction
data_condition_version <- data %>%
  group_by(Player_ID, Phrase_Condition, Game_Version) %>%
  summarise(Average_Accuracy = mean(Answer), .groups = 'drop')

#This doesnt work:
#model4 <- lmer(Average_Accuracy ~ 1 + Phrase_Condition * Game_Version + (1 + Phrase_Condition | Player_ID), data = data_condition_version)
model4 <- lmer(Average_Accuracy ~ 1 + Phrase_Condition * Game_Version + (1 | Player_ID), data = data_condition_version)

# Obtain and print a summary of the model to examine fixed and random effects
print(summary(model4))
## Linear mixed model fit by REML. t-tests use Satterthwaite's method [
## lmerModLmerTest]
## Formula: Average_Accuracy ~ 1 + Phrase_Condition * Game_Version + (1 |  
##     Player_ID)
##    Data: data_condition_version
## 
## REML criterion at convergence: -360.5
## 
## Scaled residuals: 
##     Min      1Q  Median      3Q     Max 
## -3.5947 -0.3806  0.1822  0.5451  1.9247 
## 
## Random effects:
##  Groups    Name        Variance Std.Dev.
##  Player_ID (Intercept) 0.000692 0.02631 
##  Residual              0.002653 0.05150 
## Number of obs: 152, groups:  Player_ID, 38
## 
## Fixed effects:
##                                   Estimate Std. Error         df t value
## (Intercept)                       0.961806   0.019278 120.524055  49.892
## Phrase_Condition2                -0.027637   0.024280 102.000000  -1.138
## Phrase_Condition3                 0.002905   0.024280 102.000000   0.120
## Phrase_Condition4                -0.222700   0.024280 102.000000  -9.172
## Game_Version2                    -0.003531   0.027263 120.524055  -0.130
## Game_Version3                     0.008912   0.026573 120.524055   0.335
## Game_Version4                     0.003053   0.026573 120.524055   0.115
## Phrase_Condition2:Game_Version2  -0.021755   0.034336 102.000000  -0.634
## Phrase_Condition3:Game_Version2  -0.080703   0.034336 102.000000  -2.350
## Phrase_Condition4:Game_Version2  -0.012499   0.034336 102.000000  -0.364
## Phrase_Condition2:Game_Version3  -0.029769   0.033467 102.000000  -0.890
## Phrase_Condition3:Game_Version3  -0.066345   0.033467 102.000000  -1.982
## Phrase_Condition4:Game_Version3  -0.068849   0.033467 102.000000  -2.057
## Phrase_Condition2:Game_Version4  -0.025387   0.033467 102.000000  -0.759
## Phrase_Condition3:Game_Version4  -0.018297   0.033467 102.000000  -0.547
## Phrase_Condition4:Game_Version4  -0.046639   0.033467 102.000000  -1.394
##                                 Pr(>|t|)    
## (Intercept)                      < 2e-16 ***
## Phrase_Condition2                 0.2577    
## Phrase_Condition3                 0.9050    
## Phrase_Condition4               5.52e-15 ***
## Game_Version2                     0.8972    
## Game_Version3                     0.7379    
## Game_Version4                     0.9087    
## Phrase_Condition2:Game_Version2   0.5278    
## Phrase_Condition3:Game_Version2   0.0207 *  
## Phrase_Condition4:Game_Version2   0.7166    
## Phrase_Condition2:Game_Version3   0.3758    
## Phrase_Condition3:Game_Version3   0.0501 .  
## Phrase_Condition4:Game_Version3   0.0422 *  
## Phrase_Condition2:Game_Version4   0.4499    
## Phrase_Condition3:Game_Version4   0.5858    
## Phrase_Condition4:Game_Version4   0.1665    
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Correlation matrix not shown by default, as p = 16 > 12.
## Use print(summary(model4), correlation=TRUE)  or
##     vcov(summary(model4))        if you need it
#Diagnostic plots
plot_residuals_diagnostics(model4)

## `geom_smooth()` using formula = 'y ~ x'

# Diagnostic checks - Simulating residuals to validate model assumptions
residuals_simulation <- simulateResiduals(fittedModel = model4, n = 500)
plot(residuals_simulation)

# Model diagnostics
check_model(model4)

# Check for influential cases potentially affecting the model
#influence_measures <- influence(model4)
#plot(influence_measures, which = "cook")

# If assumptions are not met, consider re-fitting the model with transformations or different specifications
# model_revised <- update(model, . ~ . + log(Reaction_Time))
# print(summary(model_revised))

# Model interpretation - examining fixed and random effects
fixed_effects <- fixef(model4)
random_effects <- ranef(model4)

# Printing fixed effects for interpretation
print(fixed_effects)
##                     (Intercept)               Phrase_Condition2 
##                     0.961805556                    -0.027637486 
##               Phrase_Condition3               Phrase_Condition4 
##                     0.002905329                    -0.222699519 
##                   Game_Version2                   Game_Version3 
##                    -0.003531444                     0.008912037 
##                   Game_Version4 Phrase_Condition2:Game_Version2 
##                     0.003053124                    -0.021755046 
## Phrase_Condition3:Game_Version2 Phrase_Condition4:Game_Version2 
##                    -0.080703094                    -0.012498578 
## Phrase_Condition2:Game_Version3 Phrase_Condition3:Game_Version3 
##                    -0.029769227                    -0.066344514 
## Phrase_Condition4:Game_Version3 Phrase_Condition2:Game_Version4 
##                    -0.068849222                    -0.025387477 
## Phrase_Condition3:Game_Version4 Phrase_Condition4:Game_Version4 
##                    -0.018297161                    -0.046639106
# Printing random effects for interpretation
print(random_effects)
## $Player_ID
##      (Intercept)
## 1   0.0055429197
## 2  -0.0042927426
## 3   0.0191750711
## 4   0.0080938598
## 5  -0.0040876506
## 6  -0.0209085239
## 7   0.0051523019
## 8   0.0052127448
## 9  -0.0138879801
## 10  0.0015230378
## 11  0.0041825285
## 12 -0.0169229170
## 13  0.0108699280
## 14  0.0094571851
## 15  0.0251881424
## 16 -0.0186290065
## 17  0.0137104974
## 18 -0.0293793958
## 19 -0.0279570021
## 20 -0.0084522419
## 21 -0.0097445849
## 22  0.0112085579
## 23  0.0054259932
## 24 -0.0285942721
## 25  0.0262337031
## 26  0.0053075448
## 27  0.0012614904
## 28  0.0253108116
## 29  0.0287031571
## 30  0.0152586747
## 31 -0.0031152626
## 32  0.0002546655
## 33  0.0096590268
## 34  0.0101938355
## 35  0.0180955406
## 36  0.0031894399
## 37 -0.0444359890
## 38 -0.0378030886
## 
## with conditional variances for "Player_ID"
# Conducting pairwise comparisons of phrase conditions using estimated marginal means
emm_results <- emmeans(model4, specs = pairwise ~ Phrase_Condition)
## NOTE: Results may be misleading due to involvement in interactions
print(summary(emm_results))
## $emmeans
##  Phrase_Condition emmean      SE  df lower.CL upper.CL
##  1                 0.964 0.00939 121    0.945    0.983
##  2                 0.917 0.00939 121    0.898    0.936
##  3                 0.925 0.00939 121    0.907    0.944
##  4                 0.709 0.00939 121    0.691    0.728
## 
## Results are averaged over the levels of: Game_Version 
## Degrees-of-freedom method: kenward-roger 
## Confidence level used: 0.95 
## 
## $contrasts
##  contrast                              estimate     SE  df t.ratio p.value
##  Phrase_Condition1 - Phrase_Condition2  0.04687 0.0118 102   3.961  0.0008
##  Phrase_Condition1 - Phrase_Condition3  0.03843 0.0118 102   3.248  0.0084
##  Phrase_Condition1 - Phrase_Condition4  0.25470 0.0118 102  21.525  <.0001
##  Phrase_Condition2 - Phrase_Condition3 -0.00843 0.0118 102  -0.713  0.8918
##  Phrase_Condition2 - Phrase_Condition4  0.20783 0.0118 102  17.565  <.0001
##  Phrase_Condition3 - Phrase_Condition4  0.21627 0.0118 102  18.277  <.0001
## 
## Results are averaged over the levels of: Game_Version 
## Degrees-of-freedom method: kenward-roger 
## P value adjustment: tukey method for comparing a family of 4 estimates